Hypertext Categorization using Hyperlink Patterns and Meta Data

Hypertext poses new text classi cation research challenges as hyperlinks, content of linked documents, and meta data about related web sites all provide richer sources of information for hypertext classi cation that are not available in traditional text classi cation. We investigate the use of such information for representing web sites, and the e ectiveness of di erent classi ers (Naive Bayes, Nearest Neighbor, and Foil) in exploiting those representations. We nd that using words in web pages alone often yields suboptimal performance of classi ers, compared to exploiting additional sources of information beyond document content. On the other hand, we also observe that linked pages can be more harmful than helpful when the linked neighborhoods are highly \noisy" and that links have to be used in a careful manner. More importantly, our investigation suggests that meta data which is often available, or can be acquired using Information Extraction techniques, can be extremely useful for improving classi cation accuracy. Finally, the relative performance of the di erent classi ers being tested gives us insights into the strengths and limitations of our algorithms for hypertext classi cation.